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A  common  assumption  in  organizations  is  that  information  sharing  improves  situation 
awareness  and  ultimately  organizational  effectiveness.  The  sheer  volume  and  rapid 
pace  of  information  and  communications  received  and  readily  accessible  through 
computer  networks,  however,  can  overwhelm  individuals,  resulting  in  data  overload  from 
a  combination  of  diverse  data  sources,  multiple  data  formats,  and  large  data  volumes. 
The  current  conceptual  framework  of  network  enabled  operations  (NEO)  posits  that 
robust  networking  and  information  sharing  act  as  a  positive  feedback  loop  resulting 
in  greater  situation  awareness  and  mission  effectiveness  in  military  operations  (Alberts 
and  Garstka,  2004).  We  test  this  assumption  in  a  large-scale,  2-week  military  training 
exercise.  We  conducted  a  social  network  analysis  of  email  communications  among  the 
multi-echelon  Mission  Command  staff  (one  Division  and  two  sub-ordinate  Brigades)  and 
assessed  the  situational  awareness  of  every  individual.  Results  from  our  exponential 
random  graph  models  challenge  the  aforementioned  assumption,  as  increased  email 
output  was  associated  with  lower  individual  situation  awareness.  It  emerged  that  higher 
situation  awareness  was  associated  with  a  lower  probability  of  out-ties,  so  that  broadly 
sending  many  messages  decreased  the  likelihood  of  attaining  situation  awareness. 
This  challenges  the  hypothesis  that  increased  information  sharing  improves  situation 
awareness,  at  least  for  those  doing  the  bulk  of  the  sharing.  In  addition,  we  observed  two 
trends  that  reflect  a  compartmentalizing  of  networked  information  sharing  as  email  links 
were  more  commonly  formed  among  members  of  the  command  staff  with  both  similar 
functions  and  levels  of  situation  awareness,  than  between  two  individuals  with  dissimilar 
functions  and  levels  of  situation  awareness;  both  those  findings  can  be  interpreted  to 
reflect  effects  of  homophily.  Our  results  have  major  implications  that  challenge  the  current 
conceptual  framework  of  NEO.  In  addition,  the  information  sharing  network  was  largely 
imbalanced  and  dominated  by  a  few  key  individuals  so  that  most  individuals  in  the 
network  have  very  few  email  connections,  but  a  small  number  of  individuals  have  very 
many  connections.  These  results  highlight  several  major  growing  pains  for  networked 
organizations  and  military  organizations  in  particular. 

Keywords:  network  organization,  sociotechnical  system,  Pareto  principle,  communication  exponential  random 
graph  model,  homophily,  degree  distribution,  training  effectiveness 
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INTRODUCTION 

Advances  in  information  and  network  technology  continue 
to  transform  the  way  human  organizations  communicate  and 
operate.  This  is  evident  as  networked  organizations  are  at  the 
core  of  the  political,  military,  economic,  and  social  fabric  of 
the  twenty-first  century  (Castells,  2009).  The  same  technological 
advances  that  have  given  rise  to  networked  forms  of  organization 
also  facilitate  their  study.  For  example,  larger  and  larger  volumes 
of  data  that  characterize  our  “digital  behaviors,”  including 
communication  and  collaboration,  are  increasingly  collected  by 
companies,  governments,  and  researchers  alike  (Navaroli  and 
Smyth,  2015).  Using  this  digital  behavior  data,  organizations 
can  be  characterized  as  social  networks  with  nodes  representing 
individuals  and  links  representing  the  interactions  between 
them.  Many  such  networks  are  inherently  complex  in  the  sense 
that  their  structure  is  irregular,  task-  and  context-specific,  and 
dynamically  evolving  in  time. 

Over  the  past  decade,  the  social  sciences  have  seen  rapid 
growth  in  research  and  understanding  of  the  structure  of 
real-world  complex  networks  (Borgatti  et  al.,  2009).  However, 
the  effects  that  operating  within  such  complex  networks 
have  upon  individual  macro-cognitive  processes  is  not  well 
understood  (Klein  et  al.,  2003).  Organizations  can  confer 
considerable  advantages  to  information  sharing  as  the  number 
of  potential  collaborations  may  be  virtually  limitless,  as  is  the 
availability  of  information.  There  are  however,  some  potential 
downsides  as  well,  as  the  resulting  deluge  of  information  (Gleick, 
2012)  can  quickly  overwhelm  human  cognitive  capabilities. 
Understanding  the  relationship  between  network  structure, 
human  collaboration,  and  cognitive  work  processes  within  real 
organizations  is  a  critical  challenge.  This  is  especially  true  in 
command  and  control  domains,  such  as  military  operations, 
emergency  response,  managing  safety  critical  systems,  air  traffic 
control,  computer  network  defense  service  providers,  and  others. 
In  all  these  naturalistic  domains  information  from  various 
sources  and  of  varying  quality  must  be  quickly  assimilated 
and  shared  among  distributed  team  members  to  make  critical 
decisions  with  potentially  significant  consequences. 

A  prevalent  perspective  within  these  domains  is  that  increased 
networking  capabilities  lead  to  greater  information  sharing  and 
availability  of  information  which  ultimately  results  in  improved 
collaboration,  organizational  efficiency,  and  better  situation 
awareness  (SA).  We  explore  this  assumption,  investigating 
macro-cognitive  processes  using  data  collected  in  a  large-scale 
exercise  of  military  network  level  operations.  We  focus  on  the 
relationship  between  information  sharing  and  SA  within  a  real- 
world  networked  organization. 

Network  Enabled  Operations  and 
Information  Fusion 

The  tenets  of  network-enabled  operations  (NEO;  Alberts  and 
Garstka,  2004)  provide  an  influential  conceptual  framework 
for  understanding  how  increased  networking  affects  human 
collaboration  and  organizational  performance  within  the  military 
domain.  This  framework  posits  that  communication  and 
information  sharing  act  as  a  positive  feedback  loop  with  increased 


information  sharing  resulting  in  greater  situation  awareness 
and  mission  effectiveness  in  military  operations.  From  a  policy 
perspective,  enhancing  information  sharing  within  and  across 
organizations  has  been  and  is  a  major  priority  for  investment 
by  the  United  States  government  including  the  Department  of 
Defense  (Alberts  et  al.,  1999;  Alberts  and  Garstka,  2004),  Federal 
Emergency  Management  Agency,  Department  of  Homeland 
Security  (Department  of  Homeland  Security,  2013),  and  the 
Federal  Aviation  Administration  (2014).  As  information  sharing 
is  increasingly  promoted  within  NEO,  it  becomes  critical  to 
explore  and  understand  the  relationships  between  information 
sharing,  cognition,  and  situation  awareness  among  the  staff  in 
these  complex  operational  environments. 

The  positive  effects  of  increased  information  sharing  upon 
SA  can  be  greatly  diminished  if  individuals  reach  a  state 
of  information  overload.  A  major  tenet  of  the  Office  of 
the  Secretary  of  Defense’s  “data-to-decision”  (D2D)  initiative 
(Swan  and  Hennig,  2012),  and  a  primary  challenge  for 
military  commanders  and  their  staff  is  to  shorten  the  cycle 
time  and  improve  the  processes  of  synthesizing  data  to 
information  and  into  knowledge  to  support  decision-making 
and  action.  Organizational  performance  and  effectiveness 
are  curtailed  by  failures  or  bottlenecks  at  any  step  in 
this  D2D  sequence.  Effectively  managing  the  entire  process 
requires  broad  collaboration  and  flexibility  in  supporting 
multiple  information  and  decision  requirements.  In  networked 
organizations,  however,  the  sheer  volume  and  rapid  pace  of 
information  and  communications  received  and  readily  accessible 
from  diverse  sources  and  in  multiple  formats  can  quickly 
overwhelm  individuals  in  the  D2D  pipeline.  Well-designed 
automation  and  decision-support  tools  can  provide  some 
assistance  in  the  D2D  cycle;  however,  the  volume  of  data  flowing 
through  large  organizational  networks  is  often  beyond  the  ability 
of  current  software  tools  to  capture,  curate,  and  store  (Salimi  and 
Vita,  2006;  White,  2012)  or  to  process  the  data  within  a  tolerable 
time  frame  (Snijders  et  al.,  2012). 

A  critical  process  of  the  D2D  pipeline  is  that  of  information 
fusion.  Software  tools  and  automation  currently  lack  the 
capabilities  to  synthesize  information  in  an  adaptive,  highly 
context-aware  manner,  which  necessitates  human  involvement 
and  considerable  cognitive  resources  (Blasch  et  al.,  2011).  Many 
contextual  factors  affect  the  human  ability  to  rapidly  synthesize 
information  into  a  coherent  understanding  of  the  current 
situation,  including  information  volume,  quality,  and  modality, 
the  general  level  of  risk  and  time-pressure  in  the  environment, 
and  factors  operating  at  the  level  of  the  individual  decision¬ 
maker,  including  cognitive  load,  fatigue,  level  of  expertise,  and 
personality  traits  such  as  need  for  closure  and  need  for  cognition. 
The  concept  of  cognitive  information  fusion  (Blasch  et  al.,  2012) 
emphasizes  the  necessity  and  strength  of  the  human  element 
in  order  to  achieve  a  high-level,  contextual  understanding  of  a 
given  situation.  Data  fusion  is  a  term  typically  used  to  describe 
computational  frameworks  for  constructing  a  comprehensive, 
data  aggregation  system  that  processes  information  to  support 
user  decision-requirements  (Klein,  2004),  whereas  cognitive 
information  fusion  explicitly  emphasizes  the  need  for  human 
cognition  and  staff  collaborations  to  integrate  and  rapidly  make 
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sense  of  these  data  streams  that  are  distributed  across  space  and 
time.  The  outcome  of  proficient  cognitive  information  fusion  is 
high  situation  awareness,  which  we  describe  in  detail  below. 

Situation  Awareness  in  NEO 

Situation  Awareness  (SA)  is  defined  as  “the  perception  of  the 
elements  in  the  environment  within  a  volume  of  time  and  space, 
the  comprehension  of  their  meaning  and  the  projection  of  their 
status  in  the  near  future”  (Endsley,  1995,  p.  36).  SA  is  a  well- 
known  concept  in  a  variety  of  domains  that  require  cognitive 
information  fusion,  including  military  operations  (Endsley,  2000; 
Matthews  et  al.,  2004),  aviation  (Kaber  et  al.,  2002;  Keller  et  al., 
2004),  air  traffic  control  (Endsley  and  Kiris,  1995;  Endsley 
and  Smolensky,  1998;  Hauss  and  Eyferth,  2003),  transportation 
(Zheng  et  al.,  2004)  and  many  others.  The  three-level  model  of 
SA  proposed  by  Endsley  (1995)  is  perhaps  the  most  common 
model  of  SA  (other  models  include  those  discussed  in  Smith 
and  Hancock,  1995  and  Bedny  and  Meister,  1999).  Endsley’s 
model  depicts  SA  as  an  essential  input  into  human  decision¬ 
making  cycles  that  is  composed  of  three  hierarchical  levels: 
(level  1)  the  perception  of  the  elements  in  the  environment 
(level  2),  the  comprehension  of  their  meaning,  and  (level  3)  the 
projection  of  their  future  status.  In  the  current  work  we  use  SA 
as  a  measure  of  an  individual’s  success  at  performing  cognitive 
information  fusion  to  comprehensively  understand  the  current 
status  of  events  transpiring  on  a  simulated  battlefield. 

At  the  cognitive  or  nodal  level,  the  relationship  between 
information,  situation  awareness,  and  task  effectiveness  has  been 
extensively  investigated  in  a  number  of  ways  including  carefully 
controlled  laboratory  behavioral  experiments.  For  example, 
Gonzalez  and  Wimisberg  (2007)  demonstrated  that  practice 
effectively  improved  information  processing,  the  attainment 
of  SA,  and  performance  on  dynamic  decision  tasks.  Further, 
training  reduced  the  relationship  between  individual  cognitive 
abilities  and  SA  to  suggest  that  the  cognitive  demands  of 
maintaining  SA  are  reduced  with  practice.  Also,  a  recent 
laboratory-based  study  examining  human  performance  on 
simulated  command  and  control  tasks  found  that,  contrary  to 
expectations,  increasing  the  volume  of  task-relevant  information 
did  not  improve  task  performance,  but  instead  reduced  self- 
reported  SA,  leading  to  poorer  task  performance  (Marusich 
et  al.,  2016).  These  results  suggest  that  increasing  the  volume  of 
information,  even  when  it  is  accurate  and  task-relevant,  is  not 
necessarily  beneficial  to  decision-making  performance  and  may 
be  detrimental  to  SA  among  team  members.  Military  operations, 
however,  are  inherently  complex  human  endeavors  involving 
macro-cognitive  processes  that  cannot  be  fully  recreated  or 
studied  in  the  laboratory  (Klein  et  al.,  2003).  As  such,  it  is  unclear 
whether  these  laboratory  findings  regarding  the  effects  of  practice 
and  increasing  volumes  of  information  on  SA  also  manifest 
themselves  in  naturalistic  settings.  As  commanders  and  their 
staffs  collectively  face  difficult,  stressful,  and  dynamic  challenges 
in  managing  battlefield  operations,  we  need  to  determine  the 
effects  of  information  sharing,  cognition,  and  training  on  their 
SA  in  more  complex,  real-world  environments. 

Warfare  is  chaotic  and  extremely  complicated.  Resolving  the 
attendant  ambiguity  on  the  battlefield  is  both  a  cognitive  and 


collaborative  challenge  of  the  first  order.  In  these  situations, 
human  integration  of  networked  information  among  the  mission 
command  staff  is  essential  to  successful  military  operations.  A 
possible  way  to  reduce  the  potentially  detrimental  effects  of 
information  overload  is  to  distribute  information  processing 
tasks  across  the  network — allowing  separate  people  to  process 
and  act  upon  different  sets  of  information  (see,  Kozlowski  et  al., 
1999;  Salas  et  al.,  2008).  In  this  case,  a  broad  distribution 
of  information  and  SA  is  essential  for  NEO.  However, 
such  distribution  may  also  create  added  communication  and 
coordination  costs  as  well  as  additional  dependencies,  requiring 
each  person  in  the  network  to  maintain  awareness  of  the  dynamic 
situation  and  rely  on  the  performance  of  others.  Some  research 
in  military-relevant  field  exercises  demonstrates  a  significant 
relationship  between  SA  and  the  participants’  awareness  of  the 
information  in  the  central  nodes  of  a  team  (Saner  et  al.,  2009). 
This  result  suggests  that  SA  is  centralized  and  not  broadly 
disseminated  across  the  networked  organization  and  that  a 
person’s  role  and  position  within  an  organization  affects  and 
potentially  limits  the  level  of  shared  SA  that  can  be  achieved. 
Our  study  scales  up  the  results  of  these  studies  at  the  level  of  the 
individual  and  small  teams  to  examine  organizational  network 
levels  of  performance. 

The  focus  of  our  research  is  to  examine  and  characterize 
the  relationship  between  information  sharing  behavior  and 
the  distribution  of  SA  in  a  real-world  networked  military 
organization.  We  examine  how  collaboration  and  information 
sharing  among  a  large,  networked  mission  command  staff  affects 
the  attainment  and  distribution  of  individual  SA  across  a  2- 
week  real-world  military  exercise.  Specifically,  we  construct 
network  graphs  from  the  record  of  staff  communications 
throughout  the  exercise,  and  assess  how  the  structure  of  these 
graphs  relates  to  the  SA  of  individuals  within  the  network, 
as  well  as  how  this  relationship  evolves  over  the  course  of 
the  exercise.  Our  results  characterize  the  relations  between 
volume  of  information,  SA  and  performance  and  have  major 
implications  for  training  and  systems  design  in  NEO  domains. 
Next,  we  describe  this  training  event  and  our  data  collection  and 
analysis. 

MISSION  COMMAND  TRAINING  EXERCISE 
EVENT 

The  Mission  Command  Battle  Laboratory  at  Fort  Leavenworth, 
Kansas  conducted  a  training  event  exercise  focused  primarily 
on  the  mission  command  operations  of  staff  composed  of  a 
Division  headquarters  ( n  =  46)  and  two  subordinate  Brigade 
headquarters  ( n  =  21,  n  =  23).  Additional  units  and  staff  at 
echelons  above  and  below  the  Division  and  Brigades  participated 
in  the  training  event  exercise,  with  the  size  of  the  networked 
organization  in  excess  of  200  (n  =  213).  The  network  architecture 
and  digitized  nature  of  the  event  allowed  examination  of  staff 
communications  in  a  distributed,  network-enabled  environment. 
Below  we  describe  the  defining  characteristics  of  this  military 
organization,  and  the  nature  of  the  tasks  they  were  required 
to  complete. 
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Defining  Characteristics  of  the  Military 
Organization 

The  participants  were  active  duty  (and  in  a  few  cases 
retired):  Soldiers  and  officers  with  operational  staff  experience 
who  were  assigned  to  differentiated,  well-specified,  and  inter¬ 
dependent  roles.  Several  staffs  at  different  echelons  participated, 
including  a  functional  slide  of  a  Division  operations  center 
and  the  staffs  of  a  U.S.  Heavy  Brigade  Combat  Teams 
(mechanized)  and  a  U.K.  Coalition  Brigade  Combat  Team. 
The  units  operated  in  a  distributed  fashion  (U.S.  units  at 
Fort  Leavenworth  and  the  U.K.  unit  at  the  Land  Warfare 
Centre  in  Warminster)  over  a  communication  network  using 
specialized  military  command  and  control  hardware  and 
software.  Within  each  unit,  staff  members  carried  out  the 
duties  of  nine  different  functional  cells.  These  cells  included 
Command,  Maneuver,  Intelligence,  Fires,  Civil  Affairs,  Signal, 
Sustainment,  Protection,  and  Liaison.  Individual  responses 
and  responsibilities  to  a  given  scenario  event  in  the  training 
exercise  depended  upon  adherence  to  established  workflows 
and  standard  operating  procedures  both  within  the  unit  and 
functional  cell. 

Several  additional  small  units  and  staffs  were  presented 
in  the  exercise,  including  high  command  elements  of  an 
International  Joint  Command,  as  well  as  a  Civil  Military 
Operations  Center  to  facilitate  coordination  of  joint,  interagency 
(e.g..  Department  of  State,  United  States  Agency  for  International 
Development),  intergovernmental,  and  multinational  efforts.  In 
addition,  a  third  Infantry  Brigade  Combat  Team  was  notionally 
represented;  however,  their  area  of  operations  was  quiet  and 
not  fully  exercised  by  scenario  events.  At  the  lowest  level, 
a  number  of  key  role  positions  were  staffed  to  represent 
Battalion  level  units  in  Army  Aviation,  Engineering,  Military 
Police,  and  Sustainment  (i.e..  Counter  Improvised  Explosive 
Device).  We  used  an  electronic  survey  instrument  to  collect  SA 
information  from  the  Division,  Heavy  Brigade  Combat  Team, 
and  the  Coalition  Brigade  Combat  Team,  as  these  groups  and 
their  interoperability  were  the  primary  focus  of  the  exercise. 
The  high  command  elements,  the  Infantry  Brigade  Combat 
Team,  and  Battalion  level  units  did  not  receive  the  electronic 
survey. 

The  military  organization  was  staffed  and  convened 
specifically  to  execute  and  accomplish  a  particular  2-week 
long  training  mission.  They  worked  interdependently  and 
engaged  in  collaborative  decision-making  for  mission  planning 
and  execution.  The  organization  functioned  as  a  purposive 
social  system ,  where  members  are  readily  identifiable  to  each 
other  by  role  and  work  interdependently  to  accomplish  one 
or  more  collective  objectives  (Hackman  and  Katz,  2010).  The 
responsibility  for  performing  the  various  tasks  and  sub-tasks 
necessary  for  mission  success  is  divided  and  assigned  among 
the  staff. 

Defining  Characteristics  of  the  Tasks 

The  training  scenario  in  a  military  exercise  generates  many 
overlapping  series  of  event-driven  tasks,  the  resolution  of  which 
requires  a  high  degree  of  coordination  among  the  participating 


command  and  control  staff.  Researchers  have  long  pointed  out 
that  the  nature  of  a  task  has  a  great  influence  on  the  steps  and 
processes  a  group  uses  to  perform  the  work  (e.g.,  Roby  and 
Lanzetta,  1958;  McGrath  and  Kravitz,  1982).  The  tasks  of  groups 
in  the  military  domain  considered  here  have  four  distinguishing 
features: 

1.  Specific  Presenting  Problems:  The  military  command  and 
control  staff  is  tasked  with  addressing  specific  problems  that 
occur  in  the  unit’s  area  of  operations.  The  military  staff 
organization  must  monitor  key  events  and  successfully  plan 
and  coordinate  an  effective  response,  given  limited  resources. 
The  presenting  problems  may  be  kinetic  events,  such  as 
responding  to  an  improvised  explosive  device,  or  civil-military 
in  nature,  such  as  responding  to  a  civil  demonstration  or 
safeguarding  polling  sites  and  maintaining  a  chain  of  custody 
in  the  transfer  of  voting  ballots.  At  other  times,  the  presenting 
problem  may  be  a  time-sensitive  intelligence  report  of  enemy 
activity  that  needs  to  be  analyzed  and  corroborated.  At  any 
given  time,  the  organization  must  coordinate  a  response  to 
many  such  presenting  problems. 

2.  Adherence  to  specific  tactics,  techniques,  and  procedures:  The 
groups  adhere  to  formalized  military  work  routines  and 
processes  that  are  known  in  advance  and  involve  delegation 
of  specific  work  responsibilities  to  various  sub-groups  and 
individual  staff  members. 

3.  Addressed  immediately:  The  group  operates  in  an  urgent,  time- 
sensitive  work  environment  and  is  required  to  immediately 
coordinate  responses  to  work  events  that  may  have  adverse 
cascading  effects  if  not  addressed  in  a  timely  manner. 

4.  Results  in  collaborative  work  products  that  need  to  be 
coordinated  and  disseminated:  The  group  is  expected  to 
construct  specific,  detailed  material  products  that  will  exist 
independently  of  the  group  process  or  the  individual 
members  themselves.  For  instance,  the  Commander  and  his 
command  elements  require  regular  reports  from  the  staff 
in  order  to  achieve  situational  awareness  of  the  battlefield 
environment.  The  work  processes  themselves  and  the 
dissemination  of  both  intermediate  and  final  work  products 
occur  across  the  communication  network  as  observable 
behaviors  over  time. 

Data  Collected 

Communications  Network 

Telephone  and  email  were  two  primary  methods  of  direct 
communication  between  staff  members  during  the  exercise.  For 
each  email  message  sent  and  phone  call  made  in  our  dataset,  three 
pieces  of  information  were  automatically  logged  electronically: 
the  sender,  the  receiver,  and  the  time  of  the  communication’s 
initiation.  The  resulting  full  communications  network  consisted 
of:  (a)  an  email  network  of  213  mission  command  staff  members 
and  19168  correspondences,  and  (b)  a  telephone  network  of  3191 
calls  between  132  mission  command  staff  members.  The  survey 
methodology,  however,  was  only  applied  to  the  core  units  of 
the  Coalition  Joint  Task  Force  organization.  Thus,  a  subset  of 
the  email  communications  network  (see  Figure  1,  right  panel) 
is  subsequently  visualized  and  used  for  our  statistical  model 
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FIGURE  1  |  (A)  The  organizational  structure  of  the  Coalition  Joint  Task  Force  during  the  2-week  military  training  exercise  event  held  at  the  Mission  Command  Battle 
Laboratory  (Fort  Leavenworth,  Kansas).  The  networked  organization  spans  multiple  echelons  from  Joint  Command  to  Division  to  Brigade  to  support-Battalions. 
Communications  were  collected  for  the  entire  Coalition  Joint  Task  Force  organization.  (B)  The  core  units  exercised  during  the  training  event  consisted  of  the  Mission 
Command  staff  of  a  U.S.  Division  and  two  participating  sub-ordinate  Brigades,  a  U.S.  Fleavy  Brigade  Combat  Team  and  a  U.K.  Coalition  Brigade  Combat  Team. 
Individual  situation  awareness  data  was  collected  using  the  SAGAT  methodology  from  the  participating  staff  of  these  three  core  units. 


analysis  of  information  and  situation  awareness.  The  telephone 
network  was  sparse  and  did  not  fully  represent  all  the  members  of 
the  core  staff  and  thus  not  subjected  to  statistical  model  analysis. 

Situation  Awareness  Global  Assessment  Technique 
(SAGAT) 

A  valid  and  reliable  method  for  assessing  SA  is  essential  for 
understanding  whether  information  sharing  behavior  improves 
the  SA  of  the  personnel  involved  in  the  networked  organization. 
Techniques  such  as  the  Situation  Awareness  Global  Assessment 
Technique  (SAGAT;  Endsley,  1995)  and  the  Situation  Awareness 
Rating  Technique  (SART;  Taylor,  1990)  have  been  applied  in  a 
number  of  organizational  settings  including  military  operations 
(Salmon  et  al.,  2006),  medical  care  environments  (Wright  et  al., 
2004),  robot  control  (Chen  et  al.,  2011),  and  industrial  processes 
(Patrick  et  al.,  2007). 

In  our  electronic  survey,  we  used  the  SAGAT,  a  widely 
used  and  validated  SA  measure  (Endsley,  2000;  Sonnenwald 
and  Pierce,  2000)  that  makes  use  of  a  pop  quiz  memory  probe 
technique  to  immediately  present  a  set  of  questions  to  an 
individual  regarding  the  state  of  their  current  task  environment. 
The  SAGAT  methodology  freezes  the  event  to  assess  individual 
SA  using  targeted  sets  of  online  queries  (multi-item  quiz).  The 
SAGAT  methodology  was  developed  and  administered  twice 
daily  using  online  queries;  at  two  predetermined  times  each  day, 
an  electronic  questionnaire  popped  up  on  the  computer  monitor 
of  each  Mission  Command  staff  member.  After  completing 
the  questionnaire  and  submitting  their  responses,  the  Mission 
Command  training  exercise  resumed. 

Implementing  the  SAGAT  requires  advanced  knowledge  of 
the  events  so  that  a  targeted  set  of  queries  can  be  developed 


and  administered  to  the  participating  Mission  Command  staff. 
Each  set  of  SA  questions  was  determined  in  consultation  with 
the  lead  mission  planner  coordinating  the  training  exercise  event, 
who  determined  the  best  times  to  administer  the  SA  queries. 
Significant  mission  events  that  were  expected  to  occur  prior 
to  the  query  time  were  identified  and  questions  that  would 
assess  SA  on  these  relevant  events  were  selected.  The  questions 
were  developed  from  an  SA  requirements  analysis  conducted 
for  various  Army  Mission  Command  staff  positions  using  goal- 
directed  task  analysis  methodology  (see  Bolstad  and  Endsley, 
2003).  During  the  event,  subject-matter  experts  tailored  the 
queries  to  the  unfolding  events  and  relevant  aspects  of  mission 
in  the  area  of  operations  for  each  Unit:  US  Division,  UK  Brigade, 
and  US  Brigade.  The  SA  queries  were  broadly  applicable,  and 
not  tailored  to  each  role.  Everyone  received  the  same  SAGAT 
queries  but  the  answers  were  unit-specific.  For  example,  the 
answers  to  the  query  “In  your  sector,  which  of  the  following 
CIVILIAN  ACTIVITIES  are  currently  occurring?”  would  be 
different  for  the  US  Brigade  and  the  UK  Brigade  based  on 
what  was  happening  in  their  area  of  operations.  Ground  truth 
was  determined  based  on  tracking  events  in  the  simulation  and 
feedback  from  subject-matter  experts  controlling  the  scenario- 
based  exercise. 

Each  individual  SA  questionnaire  included  on  average  eight 
items  from  a  total  pool  of  33  general  queries.  Unanswered 
questions  were  scored  as  incorrect.  Questions  were  scored 
based  on  the  participant’s  base  unit.  The  data  was  collected 
by  a  contracted  partner,  SA  Technologies  Inc.,  to  the  Mission 
Command  Battle  Laboratory  and  provided  to  us  in  the  aggregate 
for  week  1  and  week  2  of  the  exercise  event.  A  sample  set  of 
queries  is  given  in  Table  1. 
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TABLE  1  |  Sample  19-item  quiz  administered  to  mission  command  unit 

using  SAGAT  methodology. 

1 .  At  this  time,  the  MOST  significant  CIVILIAN  event  involves  which  of  the 
following? 

2.  At  what  LEVEL  have  CYBER  ATTACKS  been  directed  against  Coalition 
operations  in  the  last  4  h? 

3.  Do  you  currently  have  troops  in  contact? 

4.  Has  a  Commander’s  Critical  Information  Requirement  been  reported  in  the 
LAST  4  h? 

5.  Have  you  received  ACTIONABLE  INTEL  in  the  last  4  h  REGARDING  High 
Value  Targets  in  your  Area  of  Operations? 

6.  How  LONG  has  it  been  since  the  last  MEDEVAC  in  your  Area  of  Operations? 

7.  In  which  portion  of  the  Area  of  Operations  was  the  LAST  CALL  for  FIRES? 

8.  In  your  sector,  which  of  the  following  CIVILIAN  ACTIVITES  are  currently 
occurring? 

9.  The  LAST  REQUEST  in  your  Area  of  Operations  from  CIVILIAN  leaders  was 
for  which  of  the  following? 

10.  The  MOST  RECENT  DETAINEES  in  your  sector  were  engaged  in  which  of 
the  following  BEFORE  CAPTURE? 

1 1 .  What  is  the  NATURE  of  the  most  recent  REQUEST  from  COALITION/HOST 
nation  partners? 

12.  What  type  of  targets  will  Counter  Coalition  Forces  attack  within  the  NEXT 
2  h? 

13.  What  was  the  COALITION  RESPONSE  to  the  last  attack  in  your  sector? 

14.  What  was  the  NATURE  of  the  last  incident  reported? 

1 5.  What  WEAPONS  did  the  Counter  Coalition  Forces  employ  in  the  LAST 
attack  in  YOUR  SECTOR? 

1 6.  Which  of  the  following  best  describes  the  TARGET  of  the  last  Counter 
Coalition  Forces  attack  in  your  sector? 

1 7.  Which  of  the  following  describe  the  OUTCOME  of  the  last  attack  in  your 
sector? 

1 8.  Which  of  the  following  have  been  INCORRECT  or  EXAGGERATED  in  media 
reports  in  the  last  4  h? 

1 9.  Which  of  these  INFRASTRUCTURE  SERVICES  are  disrupted  in  your  Area  of 
Operations? 


RESULTS 

Social  Network  Visualization 

A  network  is  defined  as  a  set  of  nodes  and  the  connections 
between  them,  called  edges  (undirected)  or  arcs  (directed). 
In  our  organizational  network  of  military  command  staff, 
the  social  collaborations  are  represented  as  directed  email 
connections  between  individual  nodes.  The  strength  of  a 
connection — number  of  email  correspondences  between  nodes — 
is  represented  by  the  thickness  of  the  line.  At  aggregate  levels 
of  analysis,  the  nodes  can  be  grouped  into  units  and  cells 
to  understand  functional  information  flows.  There  were  45 
individuals  in  Division  roles,  23  in  U.K.  brigade  roles,  and  21 
in  U.S.  Brigade  roles,  for  a  total  of  89  nodes  that  were  used 
in  both  our  network  visualization  and  subsequent  statistical 
model  and  analysis.  The  pattern  of  email  communications 
highlights  the  complex  interdependencies  and  information 
sharing  among  the  Mission  Command  staff  (Figure  2B)  and  the 
diverse  information  flows  between  functional  cells.  The  layout  of 
the  network  visualization  was  produced  using  Gephi — an  open- 
source  network  analysis  and  visualization  software  package — and 
is  energized  to  minimize  the  overall  variation  in  line  length 


using  the  Force  Atlas  (Jacomy,  2009)  algorithm.  This  algorithm 
effectively  centralizes  the  most  highly-connected  nodes  and 
pushes  the  least  connected  nodes  to  the  periphery.  Our 
levels  of  analysis  extend  from  the  unit-level  (e.g..  Division, 
Brigade,  and  Battalion)  to  function-cell  (Command,  Maneuver, 
Intelligence,  etc.)  all  the  way  down  to  characterizing  individual 
staff  members.  The  network  visualization  highlights  the  sheer 
complexity  of  current  information  sharing  environments  to 
achieve  coordination  and  unity  of  effort  among  the  Mission 
Command  staff. 

Imbalanced  Information  Sharing 

The  distribution  of  email  communications  among  the  command 
staff  is  represented  by  in-degree  and  out-degree.  The  in-degree 
of  a  node  is  the  number  of  individuals  who  send  messages 
to  that  node.  Conversely,  the  out-degree  is  the  number  of 
individuals  who  receive  messages  from  that  node.  In  our 
observed  mission  command  network,  a  fundamental  asymmetry 
exists  in  the  degree  that  distribution  of  information  flows 
among  staff  email  collaborations.  A  few  key  individuals  dominate 
information  sharing  among  the  staff.  This  is  apparent  in  the 
cumulative  distributions  of  in-degree  and  out-degree  of  email 
correspondences  (see  Figures  3A,B).  These  plots  show  the 
number  of  individuals  with  degree  greater  than  or  equal  to 
a  specified  value.  Most  individuals  in  the  network  have  very 
few  connections,  but  a  small  number  of  individuals  have  many 
connections.  Steeper  drop-offs  in  these  plots  correspond  to 
greater  asymmetry  in  the  degree  distribution.  The  dominance 
of  key  members  of  the  Mission  Command  staff  conforms  to 
a  general  network  property  of  complex  systems.  The  degree 
distributions  of  real-world  networks  are  typically  skewed  and 
non-normal  (i.e.,  non-Gaussian)  with  heavy  tails  (Barabasi  et  al., 
1999;  Strogatz,  2001).  Heavy-tailed  distributions  are  so  pervasive 
in  real-world  networks — turning  up  again  and  again  in  a  wide 
variety  of  both  natural  and  social  phenomena,  from  earthquakes 
and  floods  to  wealth,  talent,  and  Internet  behavior  (West,  2012) 
that  in  organizational  settings  this  phenomenon  is  known  as 
Pareto’s  Law  of  the  vital  few  (20%)  and  the  trivial  many  (80%). 
At  the  macro  level,  Pareto  (1897)  first  described  imbalances  in 
the  wealth  distribution  of  western  countries  such  that  20%  of  the 
people  owned  80%  of  the  wealth.  The  seminal  importance  of  this 
pioneering  work  is  noted  by  West  (2012,  p.  78),  who  describes 
Pareto  as  “the  first  to  have  the  modern  vision  of  society  as  a 
network  of  reciprocal  and  mutually  interdependent  entities.” 

In  our  email  communication  network,  key  individuals  at 
the  tail  of  the  degree  distribution  were  found  to  dominate 
collaborations.  The  steeper  drop-off  of  Figure  3B,  as  well  as 
the  more  extended  tail  indicates  that  this  was  even  more 
evident  in  the  out-degree  distribution  than  the  in-degree 
distribution.  We  examine  these  degree  imbalances  in  terms  of 
the  Pareto  phenomenon  (see  Figures  3C,D).  Degree  rank  is 
plotted  on  the  x-axis,  with  1  being  the  individual  with  the 
highest  degree,  and  the  percentage  of  all  in-degree  connections 
in  the  network  belonging  to  that  individual  is  plotted  on 
the  y-axis.  Here  again,  a  steeper  curve  indicates  a  greater 
imbalance  in  the  degree  distribution.  We  mark  on  the  curves 
the  points  denoting  how  many  individuals  are  responsible  for 
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FIGURE  2  |  Network  visualization  of  email  communications  between  the  Mission  Command  staff  across  a  2-week  training  exercise  event 
encompassing  two  echelons  of  Command— a  Division  and  two-subordinate  Brigades.  Email  communications  are  aggregated  at  the  cell  level  to  reveal 
functional  cell-to-cell  correspondences  (A)  and  disaggregated  at  individual  node  level  (B).  Node  color  indicates  functional  cell  assignments  for  all  members  of  the 
Mission  Command  staff,  which  are  specified  in  the  legend.  The  color  and  thickness  of  the  lines  denote  the  functional  cell  of  the  sender  and  message  volume. 


FIGURE  3  1  In-degree  (A)  and  out-degree  (B)  cumulative  distribution  functions  for  the  full  email  communications  network.  Such  heavy-tailed  distributions 
are  common  in  complex  networks.  The  dominance  of  some  members  of  the  Mission  Command  staff  is  evident  when  expressed  as  a  percentage  of  all  ties  (email 
connections)  for  in-degree  (C)  and  out-degree  (D).  The  inserted  lines  show  the  percentage  of  nodes  that  subsume  80%  of  the  in-ties  or  out-ties. 


80%  of  the  ties.  In  the  in-degree  distribution,  44%  of  the 
staff  were  responsible  for  80%  of  the  in-ties.  In  the  out- 
degree  distribution,  only  31%  of  the  nodes  were  responsible 
for  80%  of  the  out-ties,  nearing  the  classic  Pareto  distribution. 
Ultimately,  this  is  interpretable  as  the  implicit  imbalance 
and  pervasiveness  of  heavy-tailed  distributions  in  complex 
networks. 


Exponential  Random  Graph  Statistical 
Model 

Exponential-family  random  graph  models  (ERGMs)  are  a  family 
of  statistical  models  widely  used  for  inferential  analysis  of  social 
network  data  (Hunter  et  al.,  2008).  Observed  networks  are 
standalone  instances  of  many  possible  realizations  of  a  given 
network.  To  support  statistical  inference  about  the  structure 
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of  a  given  network,  an  ERGM  compares  the  similarity  of  the 
current  observed  network  to  the  set  of  all  possible  alternative 
configurations.  This  allows  us  to  establish  a  statistical  baseline 
to  infer  the  likelihood  that  the  network  could  have  expressed 
the  observed  structural  characteristics  at  random.  The  ERGM 
models  described  below  give  the  probability  of  observing  a 
particular  structural  edge — an  email  connection — as  a  function 
of  the  model  parameters,  which  are  based  on  a  variety  of  statistics 
from  the  network.  The  coefficients  are  not  unlike  those  in  a 
logistic  regression,  and  can  be  interpreted  as  their  effect  on  the 
log-odds  of  observing  a  given  edge.  In  the  email  network,  for 
example,  the  log-odds  of  observing  an  edge  that  reciprocates 
another  edge  is  significantly  higher  than  observing  an  edge  that 
does  not  reciprocate  an  edge. 

Using  the  ergm  package  in  R  (Handcock  et  al.,  2016),  we 
fit  separate  ERGM  models  to  Week  1  and  Week  2  of  the 
exercise  (Appendix).  The  model  coefficients  for  each  week  are 
plotted  in  Figure  4.  Results  that  are  positive  and  statistically 
significant  are  colored  red,  results  that  are  negative  and 
significant  are  colored  blue,  and  results  that  are  not  statistically 
significant  are  shaded  black.  The  circle  represents  the  value  of 
the  coefficient  and  the  lines  represent  the  accompanying  95% 
confidence  interval.  Due  to  the  sheer  number  of  communications 
in  our  dataset,  some  model  coefficients  have  very  small  but 
significant  effects  even  though  they  appear  to  sit  on  the  0 
mark.  We  describe  the  effective  terms  of  the  statistical  model  in 
detail  below. 


Robust  Information  Sharing  Environment 

Across  both  weeks,  we  find  strong  positive  effects  for  within-cell 
homophily,  reciprocity,  triadic  closure,  and  indegree.  Elomophily 
refers  to  the  observation  that  networks  often  foster  connections 
based  on  similarity  (McPherson  et  al.,  2001);  in  our  case,  defined 
as  other  individuals  of  the  same  functional  cell  (see  Figure  2). 
These  functional  cells  are  well-defined  and  known  according  to 
the  general  staff  system  (Department  of  the  Army  Headquarters, 
2015)  and  include:  command,  maneuver,  intelligence,  fires,  civil 
affairs,  signal,  sustainment,  protection,  and  liaison.  During  both 
weeks  the  model  demonstrates  a  propensity  for  within-cell  ties 
in  the  communication  network.  Reciprocity  in  directed  email 
communications  between  two  individuals  (dyads)  refers  to  the 
likelihood  of  mutual  connections  or  email  exchanges  between 
them.  We  found  a  high  propensity  for  reciprocity  of  email 
exchanges  between  individuals.  That  is,  in  a  directed  graph, 
if  individual  A  sends  email  to  individual  B  there  is  a  strong 
likelihood  that  B  also  sends  an  email  to  A.  More  elaborate 
social  structures  arise  when  considering  three  individuals  (triads) 
since  a  much  wider  set  of  interactions  is  possible  among  them. 
Triadic  closure  refers  to  a  property  of  social  networks  that  if 
relations  exist  between  two  pairs  of  individuals  (A-B  and  A-C), 
then  there  is  a  strong  likelihood  of  a  tie  (B-C)  that  completes 
the  triangle  of  relations.  Both  reciprocity  and  triadic  closure  are 
common  features  of  social  networks  (Scott,  2012).  The  model 
terms  indegree,  outdegree,  and  triadic  closure  were  geometrically 
weighted  to  control  for  preferential  attachment  effects  so  that 


ERGM:  Email  Network,  Week  1  ERGM:  Email  Network,  Week  2 


Coefficient  Confidence  Intervals  (95%)  Coefficient  Confidence  Intervals  (95%) 

FIGURE  4  |  Exponential  random  graph  statistical  models  of  the  email  communication  network  during  week  1  (left  panel)  and  week  2  (right  panel)  of 
the  Mission  Command  training  event  exercise.  The  models  describe  the  probability  of  observing  any  given  edge  as  a  function  of  the  coefficients  (log  odds)  in  the 
statistical  model.  Results  that  are  positive  and  significant  are  colored  red,  results  that  are  negative  and  significant  are  colored  blue,  and  results  that  are  not  statistically 
significant  are  colored  black.  The  circle  represents  the  statistical  coefficient  while  the  lines  represent  the  95%  confidence  interval  for  the  coefficient.  Note  that  given  the 
large  volume  of  messages  some  nodes  have  very  small  and  significant  effects  even  though  they  appear  to  be  sitting  on  the  0  mark. 
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each  additional  shared  partner  has  a  declining  positive  impact 
on  the  probability  of  one  or  two  persons  forming  a  tie.  This  has 
been  shown  to  work  well  in  overcoming  model  degeneracy  effects 
(i.e.,  bimodality)  and  in  producing  generalizable  models  that 
accommodate  source  and  sink  effects  (see  Hunter  and  Handcock, 
2006;  Hunter,  2007). 

Our  model  also  examines  the  association  between  tie 
formation  and  the  number  of  messages  sent  or  received, 
independent  of  degree.  Across  both  weeks  we  find  a  positive, 
significant  effect  for  the  number  of  sent  messages  and  out-degree 
[Msg.  sent  (out- ties)]  and  also  between  the  number  of  received 
messages  and  in-degree  [Msg.  received  (in-ties)].  As  the  volume 
of  messages  sent  or  received  increases,  so  does  the  number  of 
channels  through  which  the  individual  sends  or  receives  those 
messages.  That  is,  rather  than  continuing  to  direct  messages 
to  a  single  partner  or  small  set  of  partners,  an  individual  who 
sends  many  messages  is  likely  to  send  those  messages  to  a  large 
population  of  alters.  The  same  is  true  for  incoming  messages.  For 
a  separate  treatment  of  this  dataset,  see  Marusich  and  Buchler 
(2016)  for  a  detailed  account  and  model  of  the  overall  email 
communication  time  series  (by  volume)  and  how  it  relates  to  an 
external  work  variable — the  occurrence  of  significant  simulated 
scenario  events  during  the  training  event  exercise. 

Information  Sharing  and  Decreased  Situation 
Awareness 

Our  central  hypothesis  is  based  on  the  tenets  of  NEO.  Alberts 
and  Garstka  (2004)  posit  that  increased  information  sharing 
in  an  organization  improves  situation  awareness.  The  model 
coefficients  for  a  link  between  situation  awareness  and  in-degree 
[SA  (in-ties)]  or  out-degree  [SA  (out-ties)]  examine  whether 
nodes  with  higher  or  lower  situation  awareness  are  more  or  less 
likely  to  send  or  receive  ties.  For  Week  1,  we  obtained  null  results: 
the  statistical  coefficients  were  not  significant  as  tie  formation 
(number  of  in-ties  and  out-ties)  was  not  associated  with  higher 
(or  lower)  levels  of  situation  awareness  among  the  mission 
command  staff.  In  Week  2,  however,  we  do  find  a  relationship 
between  SA  and  the  propensity  to  receive  and  form  network  ties. 
Higher  SA  was  associated  with  a  higher  tendency  to  form  in¬ 
ties  and  a  lower  tendency  to  form  out-ties.  This  challenges  the 
hypothesis  that  increased  information  sharing  improves  situation 
awareness.  The  model  coefficient  for  email  in-ties  and  situation 
awareness  was  positive  and  significant,  which  indicates  that  those 
with  high  SA  were  more  likely  to  be  the  recipients  of  ties. 
Receiving  email  (in-ties)  implies  a  requirement  for  information, 
suggesting  involvement  in  an  organizational  work  process.  On 
the  other  hand,  sending  email  (out-ties)  can  be  more  material 
as  it  more  directly  advances  an  organizational  work  process 
especially  if  the  information  is  processed  and  enhanced  (value- 
added)  and  not  just  passed  along.  In  other  words,  sending  email 
is  by  definition  an  active  process  whereas  receiving  email  is  a 
passive  process.  Situation  awareness  is  usually  associated  with 
an  active  process  of  constructing  a  mental  model  of  the  current 
events  (Endsley,  2000),  and  thus  should  be  associated  with  active 
work  processes  such  as  processing  and  sending  email. 

A  plausible  explanation  of  these  results  is  that  lower  situation 
awareness  is  associated  with  work  demands.  The  implication 


is  that  sending  email  demands  attentional  resources  from  the 
user  and  thus  detracts  from  their  overall  situation  awareness 
due  to  multi-tasking  demands — in  the  same  way  that  chatting 
with  a  passenger  might  distract  a  driver  from  paying  attention 
to  the  route.  An  alternative  explanation  is  that  processing  and 
sending  email  is  associated  with  addressing  specific  challenges 
and  fashioning  work  products,  so  that  attention  is  not  broadly 
allocated  and  instead  tightly  constrained  and  focused  intently 
on  processing  a  subset  of  features  and  events  in  the  battlespace. 
A  plausible  explanation  for  the  in-tie  effect  is  that  people  with 
greater  knowledge  are  likely  to  be  tapped  as  potential  sources 
of  information  and  expertise,  per  transactive  memory  theory 
(Contractor  and  Monge,  2002;  Borgatti  and  Cross,  2003),  a 
rich-get-richer  effect.  Broadening  the  number  of  email  out-ties 
was  associated  with  lower  situation  awareness,  perhaps  because 
of  the  complexity  of  the  operational  environment  outpaces 
human  cognitive  capabilities.  In  our  broadly  collaborative  and 
information-rich  Mission  Command  network,  the  accumulation 
of  information  can  occur  quite  rapidly.  In  such  cases,  it  can 
be  difficult  and  time-consuming  for  the  human  operator  to 
process  relevant  information  and  support  workflows  due  to 
overwhelming  volume  of  information  and  variety  of  different 
sources  (email,  chatrooms,  maps  with  graphical  overlays, 
imagery,  video).  An  alternative  explanation  may  be  that  those 
with  higher  SA  did  not  find  it  necessary  to  reach  out  to  others 
to  obtain  mission-critical  information,  with  more  in-ties  they 
already  had  a  firm  grasp  of  their  operating  environment.  Given 
the  limits  of  causal  inference,  it  is  also  possible  that  individuals 
with  lower  SA  may  send  out  more  requests  for  information; 
disambiguating  the  direction  of  the  effect  requires  an  analysis  of 
email  content,  which  we  do  not  have. 

We  note  the  development  of  an  additional  communication 
pattern  that  was  cemented  by  week  2,  according  to  the  model 
coefficients.  During  the  second  week,  we  found  positive  effects 
for  sending  messages  and  establishing  additional  ties,  in  addition 
to  receiving  messages  and  receiving  additional  ties.  We  found 
that  individuals  with  more  incoming  ties  were  less  likely  to  send 
larger  volumes  of  emails,  while  those  with  more  outgoing  ties  less 
likely  to  receive  more  messages.  Using  the  standard  terminology 
of  the  network  information  flow  perspective  (Zachary,  1977; 
Ahlswede  et  al.,  2000),  taken  together  these  four  effects  suggest 
that  certain  individuals  increasingly  act  as  sources  or  sinks  of 
information  in  the  networked  organization.  That  is,  they  acted  as 
either  broadcasters  or  attractors  of  information.  This  reinforces 
the  primacy  of  our  earlier  result  that  information  is  not  shared 
equally  in  the  network  with  Pareto-type  imbalances  to  the  in¬ 
degree  and  out-degree  distributions. 

Homophily  in  Email  Communications 

Homophily  effects  offer  further  insight  into  the  pattern  of 
SA  that  we  observe  in  the  network.  During  the  second  week 
we  find  a  significant,  negative  effect  for  SA  heterophily.  That 
is,  individuals  with  larger  differences  in  SA  are  less  likely  to 
form  ties  with  one  another  (i.e.,  lower  SA  individuals  tend  to 
communicate  among  themselves  and  higher  SA  individuals  tend 
to  communicate  among  themselves).  This  is  another  emergent 
property  of  the  network,  as  we  did  not  observe  this  pattern  during 
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week  1 .  Over  time  the  network  appears  to  become  stratified  with 
respect  to  SA.  This  could  be  the  result  of  deliberate  action — 
individuals  with  higher  SA  may  reach  out  to  others  with  high 
SA  while  avoiding  those  with  lower  SA — or  this  may  be  an 
outcome  of  the  structural  configuration  of  the  network — those 
with  access  to  information  that  enhances  SA  were  unable  to 
diffuse  that  information  to  parts  of  the  network  and,  as  a  result, 
SA  declined  among  those  subgroups.  The  stratification  of  SA 
in  this  network  is  problematic  for  organizational  performance 
and  this  problem  deserves  further  attention  both  to  advance 
organizational  research  and  improve  the  effectiveness  of  military 
performance  during  training  exercises.  One  possibility  is  that  the 
organization  is  essentially  divided  into  information  processors 
whose  job  it  is  to  understand  the  situation  and  who  receive 
and  send  an  inordinate  number  of  email  messages  (and  have 
high  SA  as  a  result)  and  other  members  of  the  staff  whose 
job  is  much  more  delimited  and  circumscribed  to  particular 
tasks  and  thus  send  and  receive  fewer  email  messages  (and 
have  low  SA  as  a  result).  In  other  words,  it  is  possible  that 
the  pattern  of  communications  reflects  a  division  of  labor 
that  emerged  among  the  mission  command  staff,  as  their 
functional  role  assignments  were  fixed  and  relate  to  their  chosen 
military  occupational  specialty,  an  enduring  property  of  their 
profession. 

DISCUSSION 

At  a  large-scale,  2-week  military  training  event  exercise,  we 
conducted  a  social  network  analysis  of  email  communications 
among  a  multi-echelon  mission  command  staff  to  assess 
the  commonly  held  assumption  that  increased  information 
sharing  improves  situation  awareness  among  the  staff  in 
complex  networked  operational  environments.  Results  from  our 
exponential  random  graph  models  challenge  this  assumption, 
as  we  found  that  increased  email  output  was  associated  with 
lower  individual  situation  awareness.  Conversely,  higher  SA  was 
associated  with  a  lower  probability  of  out-ties,  so  that  sending 
too  many  messages  broadly  to  other  individuals  decreased  the 
likelihood  of  attaining  SA.  This  challenges  our  hypothesis  that 
increased  information  sharing  improves  situation  awareness 
and  also  supports  a  recent  laboratory  studies  that  increasing 
task-relevant  information  did  not  improve  task  performance, 
but  instead  reduced  self-reported  SA,  leading  to  poorer  task 
performance  (Marusich  et  al.,  2016).  In  addition,  we  observed 
two  strong  effects  of  homophily  in  email  communication.  Links 
were  more  commonly  formed  between  members  of  the  command 
staff  with  similar  functions  and  levels  of  situation  awareness,  than 
between  two  individuals  with  dissimilar  functions  and  levels  of 
situation  awareness.  These  findings  have  major  implications  that 
challenge  the  current  conceptual  framework  of  NEO  (Alberts 
and  Garstka,  2004)  which  posits  that  robust  networking  and 
information  sharing  act  as  a  positive  feedback  loop  resulting 
in  greater  situation  awareness  and  mission  effectiveness  in 
military  operations.  These  and  other  results  highlight  several 
major  growing  pains  for  networked  organizations  and  military 
organizations  in  particular. 


Unequal  Information  Sharing 

The  first  growing  pain  for  organizations  is  that  information  is 
not  shared  equally,  even  in  robust  and  relatively  unconstrained 
information  sharing  environments.  In  our  observed  mission 
command  network,  there  were  large  imbalances  to  information 
sharing  as  a  few  key  individuals  dominated  information  sharing 
among  the  staff.  Most  individuals  in  the  network  have  very 
few  email  connections,  but  a  small  number  of  individuals  have 
very  many  connections.  The  dominance  of  key  members  of 
the  Mission  Command  staff  conforms  to  a  general  Pareto- 
type  network  property  of  complex  systems  (see  West,  2012). 
At  network  levels  of  interaction,  understanding  the  social  and 
cognitive  dynamics  that  give  rise  to  Pareto’s  law  constitutes  a 
fundamental  question  for  network  science  research.  Intuitively, 
it  is  possible  that  the  degree  distribution  imbalance  occurs 
whenever  there  is  a  fundamental  imbalance  in  the  value  of 
individuals  in  the  network.  In  our  mission  command  network 
the  value  of  individuals  is  reflected  by  military  rank/experience 
and  the  primacy  of  certain  functional  role-positions.  If  so,  this 
phenomenon  could  extend  beyond  military  networks  to  include 
any  workgroup  structured  using  an  organizational  hierarchy, 
especially  corporations,  bureaucracies,  departments,  and  work¬ 
groups  among  others. 

In  networked  organizations,  the  sheer  volume  and  rapid 
pace  of  information  and  communications  received  and  readily 
accessible  through  computer  networks  can  be  overwhelming 
to  individuals,  resulting  in  data  overload  from  diverse  data 
sources,  multiple  data  formats,  and  large  data  volumes.  The 
need  to  integrate  and  interpret  information  in  massive  data 
environments  and  the  macrocognitive  processes  involved  in 
fashioning  a  coherent  understanding  is  commonly  referred  to 
as  sensemaking  (Klein  et  al.,  2006).  Given  the  Pareto-type 
imbalances  to  the  email  degree  distributions,  it  is  likely  that 
some  individuals  in  the  network  are  beyond  their  functional 
cognitive  capacity  to  process  and  make  sense  of  so  much 
information.  It  is  the  case  that  in  complex  tasks,  limitations  in 
cognitive  resources  and  processes  have  been  shown  to  give  rise 
to  many  cognitive  biases  that  distort  human  decision  making 
(Lebiere  et  al.,  2013).  However,  humans  are  remarkably  resilient 
in  adapting  to  the  complexity  and  functional  limitations  of 
their  environment.  Researchers  have  documented  a  variety  of 
cognitive  strategies  and  systematically  examined  the  tradeoffs 
and  shortcuts  involved  in  overcoming  fixed  limits  to  human 
information  processing  capacities,  such  as  attention  bottlenecks 
and  memory  limitations  (see  Reitter  and  Lebiere,  2012).  One  of 
those  tradeoffs  and  associated  techniques  is  whether  to  share  raw 
information,  providing  all  the  needed  information  at  the  cost 
of  potentially  overwhelming  attentional  demands,  or  high-level 
summaries  and  conclusions,  requiring  context-sensitive  filtering 
and  inference  that  may  miss  critical  issues  in  the  presence  of 
information  stovepipes  (Tang  et  al.,  2015). 

As  a  practical  consideration,  following  the  business  maxim  put 
forward  by  Koch  (2011)  in  his  book.  The  80/20  Principle,  efforts 
should  be  made  to  support  this  vital  20%  that  also  generates 
80%  of  the  work.  This  suggests  that  technological  solutions 
and  training  regimens  should  focus  on  supporting  the  vital 
20%  of  the  networked  staff  driving  most  of  the  collaborations 
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(for  a  decision-support  agent  approach,  see  Buchler  et  al., 
2014).  The  long-tailed  distributions  of  communications  have 
major  implications  for  the  psychological  and  social  sciences  as 
many  parametric  statistical  approaches  and  human  performance 
modeling  tools  assume  some  degree  of  normality  in  the 
processes  they  model  (Warwick  et  al,  2013).  As  discussed  below, 
understanding  how  cognition  is  manifest  at  network  levels  of 
interaction  represents  a  challenge  and  opportunity  for  macro- 
cognitive  researchers. 

Scientists  and  engineers  have  developed  many  approaches 
for  understanding  and  predicting  individual  and  group  state, 
behavior,  cognition,  and  performance  in  the  context  of 
teams,  organizations,  and  societies;  with  each  approach  being 
limited  in  resolution,  validity,  and  insight  into  the  human 
condition.  Understanding  how  humans  interact  and  adapt  within 
dynamic,  complex,  natural  environments  remains  a  pressing  and 
challenging  scientific  problem.  Recent  technological  advances 
have  lead  researchers  and  information  technology  firms  (e.g., 
Navaroli  and  Smyth,  2015)  to  leverage  vast  quantities  of  data 
from  various  human,  information,  and  communication  networks 
to  make  interpretations  and  predictions  about  humans  and 
the  context  in  which  they  are  operating.  Network  science 
approaches  allow  both  the  organizational  context  and  real- 
world  human  behavior  to  be  jointly  analyzed  and  interpreted. 
However,  network  science  focuses  on  the  interactions  between 
decision-makers  and  their  emergent  social  phenomena,  often 
oversimplifying  many  cognitive  aspects  of  the  individual  nodes. 
This  represents  both  a  challenge  and  opportunity  for  macro- 
cognitive  research  to  define  cognitive  processes  that  occurs  at 
the  “nodal  level”  in  real-world  contexts,  such  as  decision-making 
under  uncertainty  and  sense-making.  In  essence,  the  defining 
challenge  is  to  understand  the  cognitive  processes  that  give  rise 
to  the  heavy-tailed  statistics  seen  at  network  levels  of  interaction. 
For  instance,  a  cognitive  mechanism  formally  implemented  at 
the  nodal  level  as  a  priority  communication  model — sorting 
communication  messages  by  importance  in  a  queue  (i.e.,  email 
inbox) — was  shown  in  simulation  to  give  rise  to  the  patterns 
of  real-world  bursty  communication  timings  observed  at  the 
network  level  (Vazquez  et  al.,  2006). 

Organizational  Stovepipes 

A  second  growing  pain  for  organizations  is  one  of  breaking  open 
“information  stovepipes”  or  existing  socio-technical  limitations 
that  restrict  the  free  flow  of  information  and  communications 
(e.g.,  Bateman,  1996).  The  flow  of  information  among  the 
Mission  Command  staff  involves  the  timely  push  and  pull  of 
information  and  knowledge  products  to  and  from  adjacent, 
higher,  and  lower  functional  cells  and  units.  The  distribution 
of  information,  however,  was  largely  constrained  to  and 
adhered  to  unit  structure  of  the  organization,  and  thus  largely 
occurred  within  functional  cell  assignments.  The  pattern  of 
communications  in  our  networked  organization  conform  to  well- 
established  principles  of  social  networks  as  we  observed  strong 
effects  of  reciprocity,  triadic  closure,  and  within-cell  homophily 
that  were  governed  by  their  functional  cell  assignment.  These 
functional  cells  are  well-defined  and  known  according  to  the 
general  staff  system  (Department  of  the  Army  Headquarters, 


2015)  and  include:  command,  maneuver,  intelligence,  fires,  civil 
affairs,  signal,  sustainment,  protection,  and  liaison.  This  is  the 
hallmark  of  a  stovepiped  organization  where  information  is 
bottled-up  and  not  widely  shared  among  diverse  individuals  in 
the  organization.  The  general  pattern  of  results  raise  fundamental 
questions  as  to  the  macro-cognitive  mechanisms  existing  at  the 
individual  node  level  that  give  rise  to  the  patterns  observed  at  the 
level  of  the  networked  organization. 

It  is  not  clear  how  to  promote  diverse  heterophilous  ties 
within  an  organization.  Currently,  two  theories  have  been 
advanced  for  a  lack  of  heterophilious  ties  in  organizational 
settings  (Chung  et  al.,  2000).  First,  rank  confers  status  within  the 
Mission  Command  network  and  higher-status  individuals  and 
organizations  in  the  multi-echelon  hierarchy  may  see  their  status 
reduced  by  ties  to  lower-status  individuals  and  organizations 
(Benjamin  and  Podolny,  1999).  In  this  case,  the  propensity  is 
to  communicate  with  high-rank  individuals.  It  is  possible  that 
this  propensity  to  concentrate  communications  to  high-ranking 
individuals  can  drive  the  types  of  in-degree  imbalances  we 
observed  in  our  email  communication  network.  Indeed,  many 
of  the  individuals  at  the  tail  of  the  degree  distribution  are  high- 
ranking  principal  members  of  the  Division  mission  command 
staff.  Second,  individuals  and  organizations  may  have  access  to 
unequal  information  quality  which  reduces  the  value  proposition 
of  information  exchanges  between  individuals  with  dissimilar 
situation  awareness.  In  addition,  maintaining  heterophilous 
relationships  across  functional  cells  and  across  unit  echelons  can 
be,  in  practice,  quite  difficult  due  to  dissimilar  work  processes, 
complex  information  requirements,  lack  of  awareness,  and  the 
multitude  of  disparate  information  systems  that  can  constrain 
such  collaborations. 

Emergence  of  Information  Sources  and 
Sinks 

A  third  growing  pain  is  the  emergence  of  individuals  that 
function  increasingly  as  sources  and  sinks  of  information  in  the 
networked  organization.  From  an  information  flow  perspective 
(Ahlswede  et  al.,  2000),  network  ties  are  social  channels  that 
allow  the  flow  of  information  throughout  the  organization.  We 
observed  that  by  Week  2  of  the  exercise,  with  more  incoming 
ties  individual  members  of  the  Mission  Command  staff  were 
less  likely  to  send  out  larger  volumes  of  emails.  With  more 
outgoing  ties,  individuals  were  also  less  likely  to  receive  more 
messages.  This  suggests  that  certain  individuals  increasingly 
act  as  sources  and  sinks  in  the  networked  organization  and 
suggests  a  specialization  of  information  sharing  behavior  as  either 
broadcasters  or  attractors  of  email  communications.  This  also 
reinforces  the  primacy  of  our  earlier  result  that  information  is 
not  shared  equally  in  the  network  with  Pareto-type  imbalances 
to  the  in-degree  and  out-degree  distributions.  Furthermore, 
these  source  and  sink  effects  are  emergent  properties  of  the 
organization.  These  results  support  earlier  research  from  military 
field  exercises  demonstrating  that  SA  is  concentrated  to  a  few 
select  individuals  and  linked  to  the  participants’  awareness  of 
the  information  in  the  central  nodes  of  a  team  (Saner  et  al., 
2009). 
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Stratified  Situation  Awareness 

A  fourth  growing  pain  was  that  over  time  our  organizational 
network  appears  to  become  stratified  with  respect  to  SA — an 
effect  of  homophily  with  respect  to  SA.  Those  with  high  situation 
awareness  were  likely  to  have  ties  to  others  who  also  have  high 
SA.  Conversely,  those  who  have  low  SA  were  likely  to  have  ties 
to  others  who  also  have  low  SA,  and  thus  have  impoverished 
information  flows.  The  effects  of  homophily  and  SA  emerged 
during  Week  2  of  the  military  training  event  exercise  and  is 
likely  a  self-reinforcing  phenomenon.  This  could  be  the  result 
of  deliberate  action — individuals  with  higher  SA  may  reach  out 
to  others  with  high  SA  while  avoiding  those  with  lower  SA — 
or  this  may  be  an  outcome  of  the  structural  configuration  of 
the  network — those  with  access  to  information  that  enhances  SA 
were  unable  to  diffuse  that  information  to  parts  of  the  network 
and,  as  a  result,  SA  declined  among  those  subgroups. 

The  stratification  of  SA  in  this  network  is  problematic  for 
organizational  performance  and  this  problem  deserves  further 
attention  both  to  advance  organizational  research  and  improve 
the  effectiveness  of  military  performance  during  training 
exercises.  One  possibility  is  that  the  organization  is  essentially 
divided  into  information  processors  whose  job  it  is  to  understand 
the  situation  and  who  receive  and  send  a  lot  of  email  messages 
(and  have  high  SA  as  a  result)  and  other  members  of  the 
staff  whose  job  is  much  more  delimited  and  circumscribed  to 
particular  tasks  and  thus  send  and  receive  fewer  email  messages 
(and  have  low  SA  as  a  result).  In  other  words,  it  is  possible  that 
the  pattern  of  communications  reflect  a  division  of  labor  that 
emerged  among  the  mission  command  staff,  as  their  functional 
role  assignments  were  fixed  and  relate  to  their  chosen  military 
occupational  specialty,  an  enduring  property  of  their  profession. 

Given  our  results,  it  is  a  likely  that  the  stratification  of 
SA  emerges  as  a  consequence  of  the  information  sharing 
behavior  of  the  organization  to  include  homophilous  ties  (and 
lack  of  heterophily)  and  Pareto-type  imbalances  in  the  degree 
distribution.  An  open  question  that  can  be  tackled  through 
simulation  is  whether  one  or  more  general  mechanisms  can 
produce  the  observed  pattern  of  results  as  an  emergent  process  of 
the  organization.  That  is,  it  is  possible  that  the  generally  observed 
properties  of  email  homophily,  reciprocity,  and  triadic  closure 
can  result  in  Pareto-type  imbalances  in  the  degree  distribution, 
which  can  in  turn  lead  to  organizational  stovepipes  among  the 
staff,  sources  and  sink  effects,  and  ultimately  the  stratification 
of  situation  awareness.  Overall,  our  result  suggests  that  SA  is 
stratified  across  the  networked  organization  and  that  a  person’s 
role  and  position  within  an  organization  affects  and  potentially 
limits  the  level  of  shared  SA  that  can  be  achieved. 

Our  approach  focused  on  relating  individual  SA  to  network 
levels  of  interaction  among  the  Mission  Command  staff.  A  more 
nuanced  approach  for  future  research  involves  defining  SA  in 
relation  to  the  information  requirements  required  for  a  given 
staff  role  position  and  unit.  Each  member  of  the  team  provides 
valuable  and  critical  information  within  and  across  roles.  For 
instance,  team  members  in  different  roles  (e.g.,  commanders, 
intelligence  officers,  logistics  officers)  have  common  information 
requirements  and  also  some  that  are  unique  to  their  functional 
role  (Artman  and  Garbis,  1998).  In  this  case,  SA  is  defined  at 
the  aggregate  team  level  and  furthermore  is  also  used  to  define 


common  or  overlapping  information  requirements  necessary  for 
shared  SA.  Although  potentially  useful  to  support  teammates, 
it  is  not  necessary  for  each  member  of  the  team  to  have  all 
the  information  needed  by  others  on  the  team.  It  is  important, 
however,  that  each  team  member  understands  what  information 
is  needed  to  support  multiple  role  positions.  Shared  SA  refers 
to  the  degree  to  which  team  members  have  the  same  SA  on 
a  defined  set  of  shared  information  requirements  (Endsley  and 
Jones,  2013).  For  effective  team  performance.  Team  SA  refers 
to  the  sum  total  of  information  and  degree  to  which  each  team 
member  obtains  the  SA  needed  to  fulfill  his  or  her  responsibilities 
(Endsley,  1995).  It  is  the  case  that  these  are  overlapping  and 
mutually  defined  sets  of  information  that  are  derived  from 
individual  SA. 

Many  of  these  challenges  faced  by  our  Mission  Command  staff 
reflect  broad  trends  and  challenges  in  networked  organizations 
and  how  to  effectively  manage  the  systematic  convergence  of 
people,  information,  and  technology  in  work-directed  networked 
organizations.  It  is  likely  that  many  of  the  findings  that  we 
observed  in  our  Mission  Command  network  are  also  evident  in 
other  organizations. 

CONCLUSION 

The  military  transformation  to  NEO  has  proceeded  under  a 
conceptual  framework  that  attempts  to  exploit  the  increasing 
interconnectedness  between  organizational  units  to  allow  more 
communication,  information  sharing,  cooperation  and  thereby 
flexibility,  adaptability,  and  mission  effectiveness  (Alberts,  2002; 
Alberts  and  Hayes,  2003).  Our  results  highlight  many  challenges 
(i.e.,  growing  pains)  to  NEO  and  the  need  for  fundamental 
research  to  guide  this  transformation;  much  of  the  rapidly 
growing  literatures  in  network  science,  organizational  and  team 
processes,  and  cognitive  science  do  not  fully  address  many  of 
the  presenting  problems  of  complex  operational  environments, 
macro-cognition,  human-in-the-loop  systems,  and  the  defining 
characteristics  of  work-driven  organizations.  The  vast  majority 
of  insights  have  been  gained  through  laboratory  research  using 
highly  controlled  contexts  and  environments.  Many  of  these 
laboratory  studies  employ  reductive  scientific  approaches  (i.e., 
divide  and  conquer)  that  do  not  scale  to  complex  real-world 
operations  or  larger  networks  and  organizational  settings.  Recent 
advances  in  technology  have  led  researchers  and  industry  to 
leverage  vast  quantities  of  data  from  various  human,  information, 
and  communication  networks  to  make  interpretations  and 
predictions  about  humans  and  the  context  in  which  they  are 
operating.  Such  “big  science”  approaches  are  fundamentally 
multi-disciplinary  endeavors  involving  teams  of  scientists  and 
engineers  that  embrace  the  complexity  of  real-world  phenomena 
to  examine  network  levels  of  interaction.  Embracing  complexity 
is  a  key  challenge  and  conceptually  is  a  paradigm-shift  for  science. 
Such  “big  science”  approaches  will  certainly  yield  fundamental 
insights  and  understanding  into  many  complex  real-world 
phenomena,  but  may  not  be  able  to  completely  predict  complex 
real-world  phenomena  that  are  non-deterministic,  non-linear, 
and  sensitive  to  initial  conditions  and  feedback  loops  (see  Arney 
et  al.,  2015). 
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Coef. 


SE 


ERGM:  Week  1 


Edges 

—4.705  *  ** 

0.305 

Within-cell  homophily 

1 .200  *  ** 

0.099 

Messages  sent  (in-ties) 

-0.001 

0.000 

Messages  sent  (out-ties) 

0.005  *  ** 

0.000 

Messages  received  (in-ties) 

0.004  *  ** 

0.001 

Messages  received  (out-ties) 

—0.002  *  ** 

0.001 

Messages  sent  heterophily 

—0.002  *  ** 

0.000 

Messages  received  heterophily 

—0.002  *  ** 

0.000 

SA  (in-ties) 

0.159 

0.384 

SA  (out-ties) 

0.081 

0.326 

SA  heterophily 

-0.163 

0.309 

Reciprocity 

2.002  *  ** 

0.153 

Triadic  closure  (GWESP) 

0.503  *  ** 

0.050 

GWESP  alpha 

1 .498  *  ** 

0.017 

GW  outdegree 

-0.439 

0.350 

GW  indegree 

2.352  *  ** 

0.652 

AIC:  3980 

BIC:  4092 

ERGM:  Week  2 

Edges 

—5.235  *  ** 

0.300 

Within-cell  homophily 

1 .105  *  ** 

0.102 

Messages  sent  (in-ties) 

—0.004  *  ** 

0.001 

Messages  sent  (out-ties) 

0.006  *  ** 

0.000 

Messages  received  (in-ties) 

0.007  *  ** 

0.001 

Messages  received  (out-ties) 

— 0.004  *  ** 

0.001 

Messages  sent  heterophily 

-0.001* 

0.000 

Messages  received  heterophily 

—0.003  *  ** 

0.001 

SA  (in-ties) 

1 .060  *  * 

0.395 

SA  (out-ties) 

— 0.989* 

0.341 

SA  heterophily 

—1 .351  *  ** 

0.331 

Reciprocity 

2.080  *  ** 

0.149 

Triadic  closure  (GWESP) 

0.750  *  ** 

0.061 

GWESP  alpha 

1 .301  *  ** 

0.016 

GW  outdegree 

0.366 

0.332 

GW  indegree 

4.535  *  ** 

0.848 

AIC:  4110 

BIC:  4223 

*  <  0.05,  **  <  0.01,  *  *  *  <  0.001. 
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